Scene Segmentation by Color and Depth Information and its Applications

Scene segmentation is a classical problem in computer vision and image processing. Traditionally this problem has been tackled by exploiting only color information from a view of the scene. Recently, a great amount of research has been done in order to develop efficient solutions for depth estimation and tools like modern stereo vision techniques and ToF range cameras allow accurate depth information extraction. The availability of both color and depth information opens the way for new insights in scene segmentation algorithms. In this paper, a novel scene segmentation framework is introduced. The goal of this approach is to capture the scene distribution, taking into account both color and depth cues, in order to provide a more accurate segmentation. Numerous applications can benefit from an effective scene segmentation method, e.g. 3D video, freeviewpoint video and game controlling. In this paper we will also show how the proposed segmentation scheme can be used to improve depth data compression performances, by using it in order to identify and extract the edges and the main objects in the scene.